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(54) Title: INVISIBLE DIGITAL WATERMARKS 
(57) Abstract 

A method and system of insertion and extraction of identification or authentication 
(watermark) data in digital media data sucn as video. The video (jifta is tiimded into blocks 
and a pseudon^dom function, such as a permutation, is applied thereto. The pennuted 
data block is then transformed using an orthogonal transform such as a Walsh Hadamaid 
Transform or a Discrete Cosine Transform. One or more of the ac coefficients generated 
by the transfoim are selected snd !ho waSATssrk data is insertdd cr^xtractsd therefrom. 
An inverse permutation and inverse transform can then be used to return the video to the 
unencoded spatial dcnnain. The inserted watennark data is substantially invisible in the 
reconstructed video since it is spread over the pixels in the block by virtue of the pennute 
and tratffifonn. 
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INVISIBLE DIGITAL WATERMARKS 

This invention relates to the provision of identification or authentication data, sometimes 
referred to as a watermark or signature, in digital media data such as digital image or audio 
5 data. In particular, die present invention relates to a method and apparatus for incorporating 
a watermark in digital media data, and a method and £q}paratus for retrieving or extracting a 
watermark from digital media data in which a watermark has been previously incorporated. 

In this specification the temi "watermark" is used to refer to any distinctive or distinguishing 
1 0 data which may be used for identification or aadientication of the digital media data associated 
therewith, or of some attribute of the media data such as the source thereof. A watermark 
may comprise image data, such as pixel data forming a logo or the like, or may be in the 
form of coded text and/or binary numbers, for exanqile, which represent a message. In some 
applications the watermark data may include error correction coding techniques to improve 
15 the robustness of the watermark to image manipulation. The format of the signal that is to 
be watermarked is not restricted to a multi dimensional representation. It is also possible for 
audio information to be watermarked. This method of encoding data is not restricted to 
information associated with copyright and could be used to convey any suitable information 
in a hidden manner. 

20 

Watermarks are utilised in media data for a number of reasons, one being to prevent or 
discourage copying of the media data if it is subject to copyright, or to at least allow for 
identification of tte media data even if it is copied. Visible watermarks have been employed 
for many years in varying applications including banknotes and photographs, but have 
25 significant disadvantages because of their visible nature. Although a visible watermark may 
be quite effective in discouraging copying of an associated image, in general it is considered 
disadvantageous for a watermark to be obtrusive upon the original image. 

Besides the issue of whetiier or not die watermark is visible in an associated image (or audible 
30 in the case of watermarked audio media), several other factors are also considered important. 
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For one, the watermark should be robust to manipulation of the watermarked media, and 
should be secure so as to not be easily removable by a malicious user. Before the advent of 
digital media processing and manipulation, a degree of robustness and security was inherent 
in a visible watermark, because a copy of the watermarked image would generally bring with 
5 it the visible watermark itself which would be difficult to remove. However, digital 
processing makes it possible to perform many sophisticated manipulative operations on 
waterniariced media, which may degrade die visible watermark or be utilised to alter an image 
to at least substantially remove the watermark. In this case, therefore, die propenies of a 
visible watermark count against the security thereof since it is clearly visible what must be 
10 removed or altered in die watermarked image. A psper entided "Protecting publicly-available 
images with a visible image watermark** (Gordon Braudaway, Karen Magerlein & Fred 
Mmtzer; SPIE Vol. 2659, pp 126-133) discusses robusmess and security in visible image 
watermarks. 

15 Visible watermarks are considered unsuitable for many modem qjplications because of the 
intrusive effect of the watermark on the original media. Watermarking schemes have been 
developed in which the watermark is substantially invisible on an original image but readily 
visible on a copy tiiereof. However, such schemes generally rely upon characteristics of 
photocopying or electronic scanning apparatus, and so are only suitable for a limited range 

20 of applications, such as in unages or text on paper documents. In any event, these 
watermarking schemes are also subject to security difficulties arising from digital processing 
and manipulation. 

In media involving a sequence of images, such as video media, it is particularly undesirable 
25 for a watermark to be intrusively visible, since considerai)le effort is expended in providing 
the image data to die user in a form which is as visually clear as possible, and a visible 
watermark may significantly detract from die original image. Visible watermarks are 
presently used in some video applications, panicularly television coverage of live sponing 
events where a relatively small and faint logo or the like is superimposed on the television 
30 picture, typically near one comer thereof. This is not completely satisfactory, besides the 
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visual intrusion, because the logo can be easily cropped from the picture in a copy thereof, 
or could be relatively easily removed, at least substantially, with digital processing 
techniques. To make the visible watermark more secure it should be placed over the visually 
most important part of the image, which also makes the watermark more intrusive and thus 
S less desirable. 

Invisible watermarking techniques, particularly for digital media data, have been developed, 
and one is described in an anicle entitied ** Watermarking Digital Images for Copyright 
Protection" (J.J.K. O'Ruanaidh, F.M. Boland & O. Sinnen). This article discloses a method 

10 of embedding a watermark in a digital image which is said to be invisible and quite robust. 
The image data is divided into rectangular blocks, and each block is then transformed using 
either a Walsh transform, discrete cosine transform (DCT) or wavelet transform. The bits 
defining the watermark graphic are inserted in the digital image by incrementing or 
decrementing a selected coefficient in the transform domain of the data block. Coefficients 

15 are selected according to a criterion based on energy content. Another algorithm described 
in tiie anicle relates to insenion of watermark data based on the use of the discrete Fourier 
transform (DFT). This method differs fundamentally from the transform domain technique 
oudined above. The DFT is a complex transform that generates complex transform domain 
coefiicients given a real valued input. The watermark is placed in the phase component of 

20 generated transform coefficients when using this transform. 

Another article which addresses the difficult issues of digital watermarking is "Secure Spread 
Spectrum Watermarking for Multimedia" (Ingemar J Cox, Joe Kilian, Tom Leighton & Talal 
Shamoon; NEC Research Institute, Technical Report 95-10). This article describes an 

25 invisible digital watermarking method for use in audio, image, video and multimedia data. 
The method described in this article also involves a frequency domain transform of the image 
data and insertion of the watermark data whilst in die transform domain. In practice, in order 
to place a lengtii n watermark into an N x N image, the discrete cosine transform of the image 
is computed, and die watermark data encoded into the n highest magnitude coefficients of die 

30 ti^fomi matrix, excluding die dc component. In other words, die watermark data is placed 
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in transform domain components of greatest perceptual significance, which enables the 
watermark to be robust to image distortion and unauthorised removal without serious 
degradation of the image itself. This watermarking algorithm employs an energy compacting 
cransfomri, which makes the selection of transform coefficients for encoding of the watermark 
S data very important. For most images the coefficients selected will be the ones corresponding 
to the low spatial frequencies, with the result that significant tampering of the image at those 
frequencies would destroy the image fidelity before the encoded watermark. The 
watermarking techniques of J.J.K O'Ruanaidh et al and Ingemar J. Cox et al require the 
original image when performing the watermark extraction operation. As a consequence, 
10 proof of ownership is accomplished only if the original image is certified as being the original 
by a trusted diird party, and the pardcular segment of the original image must be first 
identified and found before ownership is verified. 

The present invention addresses some of the difficulties identified in the prior art, and 
15 embodiments of the invention aim to provide a digital watermarking process in which: 

1. the presence of the watermark is invisible (i.e. the watermarked visual or audio 
material is visually or auditoriaily substantially indistinguishable from the original); 

20 2. the watermark is robust to signal manipulation and distortion; 

3. the watermark is secure; 

4. the original media data is not required in order to extract die watermark; and 

25 

5. the waiennark can be inserted and/or extracted by a simple computational procedure 
which can be done in real time. 

In accordance with the present invention, there is provided a method for inserting 
30 identification or audientication data into digital media data, including die steps of: 
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segmenting the digital media data into data blocks; 

applying a pseudo-random reversible funcdon to a block of the digital media data to 
obtain a modified data block; 

applying an orthogonal transform on the modified data block to obtain transform 
S domain data; 

modifying at least one selected transform domain data coefficient in accordance with 
identification or authentication data; 

inverse transforming the transform domain data having the at least one modified 
coefficient; and 

10 applying an inverse psaido-random function to obtain watermarked digital media data. 

The present invention also provides a method for extracting identification or authentication 
data from watermarked digital media data, including the steps of: 

segmenting the digital media data into data blocks; 
15 applying a pseudo-random reversible function to a block of the digital media data to 

obtain a modified data block; 

applying an orthogonal transform to the modified data block to obtam transform 
domain data; and 

extracting identification or autiientication data from at least one coefficient of the 
20 transform domain data. 

Preferably, the pseudo-random reversible function has the property of flattening the power 
spectral density of the data block (i.e. tiie function perfonms a spectral whitening operation), 
such that each coefficient tfien generated by the transform contributes substantially equally 
25 to the total energy of the block. This allows the watermarking process to be less sensitive, 
with regard to the introduced distortion, to die selection of the transform coefficient which 
is modified in the watermark insertion operation. 

The insertion and/or extraction metiiod can be performed in real time, which is particularly 
30 advantageous when the digital media data has presentation timing restrictions, such as in die 
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of real time video and/or audio data. 

It is preferred for optimal performance thai the average (dc) component of the transformed 
media data be restricted to a single known transform coefficient and that this transform 

5 coefficient is not available for modification by the watermark insertion operation. It is also 
preferred that the pseudo-random reversible function be tolerant to the introduction of noise 
resulting from signal processing that could subsequently be performed on the watermarked 
media data. Many different pseudo-random functions could be used for this application. One 
pseudo-random function that offers good performance in terms of its noise rejection 

10 capability, spectral flattening performance and simplicity of implementation is a permutation 
of the data block based upon a keyed random number generator. In that case, the user should 
ensure diat a permutation is selected that exhibits the desired ^ectral whitening characteristics 
as this is not guaranteed by all permutations. 

15 A number of different transforms exist that could be used as the orthogonal transform 
operation in the preferred method. These include the Walsh Hadamard Transform (WHT), 
Discrete Cosine Transform (DCT), Discrete Sine Transform (DST) and Fast Fourier 
Transform (FFT). The Walsh Hadamard Transform is the preferred choice due in part to its 
low implementation complexity. The AC transform coefficients generated with such a 

20 transform in conjunction with an appropriate pseudo-random function, using real image data 
as input, are characterised by all possessing approximately equal energy. The selection of 
uransform coefficient(s) for modification can tiius be based on a random keyed operation to 
further enhance die security of the watermark. 

25 For functions and transforms that do not restrict the average value of the data block to a 
single transform coefficient, it is preferred (to minimise watermark visibility) that the average 
(dc) value for the data block is calculated, stored, and subtracted from each data value in the 
data block prior to the application of the of the pseudo-random function. The average value 
is subsequently retrieved and added to each data value making up the watermarked data block 

30 immediately after the application of the inverse pseudo-random function. 



wo 98/37513 



PCT/AU98/00106 



• 7- 

The application of the pseudo-random function and the application of the onhogonal 
transform can be combined into a single operation. Similarly with respect to the inverse 
pseudo-random function and inverse transform* A combined data permutation and transform 
operation can be considered equivalent to, in the one dimensional case, performing a 
5 permutation upon the columns making up the basis matrix of the transform in question. Each 
permutation will yield an orthogonal transform, hence the number of transforms contained 
in the set is equal to the number of available permutations. Using this interpretation, the 
security of the watermark relies not just on which transform coefficient has been modified to 
contain the watermark data, but also on which member of the set of available transforms has 
10 been used. 

The present invention further provides apparatus for inserting or extracting watermark data 
in digital media data, comprising: 

segmenting means for segmenting the digital media data into data blocks; 
IS processing means for ^plying a pseudo-random reversible function to a block of the 

digital media data to obtain a modified data block and performing a transform on the modified 
data block to obtain transform domain data; and 

means for inserting or extracting watermark data in at least one coefficient of the 
transform domain data. 

20 

Preferably, in the case where watermark data is to be inserted in die digital media data the 
processing means is also adapted to perform an inverse transformation and inverse pseudo- 
random function on the transform domain data containing the watermaric data so as to obtain 
watermarked digital media data. 

25 

In prxtice, the segmenting of the digital media data into data blocks might comprise forming 
blocks of 64x64 pixels of image luminance pixel data, wtere the watermark is to be inserted 
into a still image or image sequence. The block size need not be restricted to being square 
and of dimension 64x64 pixels, both smaller and larger block sizes are possible depending 
30 upon application requirements. In practice, the identification/authentication data which is 
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inserted into a data block of digital media data might conf^)rise a pixel from a binary graphic, 
or data in the form of bits used to represent text and binary numbers, for example. The 
watemiark data is inserted into the data block that has undergone a block transform operation. 
The distortion introduced due to the insertion of watermark data is dependent upon the block 
5 size, the number of transform coefficients modified by the insertion operation and the 
magnitude of die modification. The watermark data density per block is arbitrary depending 
upon ^plication requirements. In general, however, the higher the density the more visually 
noticeable is the inserted watermark in the image, A series of data blocks may be contained 
in a single image frame or spread over a number of image frames. 

10 

The invention is described in greater detail hereinafter, by way of example only, with 
reference to the acconq)anying drawings, wherein: 

Figure 1 is a flowchart illustrating operations for inserting watermark data into digital 
media data; 

15 Figure 2 is a flowchart illustrating operations for extracting watermark data from 

digital media data; 

Figure 3 is a diagram of the watermark insertion process of a preferred embodiment 
of die present invention; 

Figure 4 is a flowchart illustrating the operations for a particular inq)lementation of 
20 the watermarking insertion procedure; 

Figure 5 is a block diagram of watermarking apparatus for real-time video; and 
Figure 6 is a block diagram of a media monitoring system. 

This invention relates to the insertion and extraction of identification or authentication data 
25 for use as a watermaric in digital media data, such as digital image data, still or sequential, 
digital audio data or the like, A watermark provided in digital media data may provide a 
means for identification of the source or some oUier attribute of the media data as may be 
required to prove copyright ownership, for example. As mentioned above, embodiments of 
the present invention are designed to have a number of advantageous properties, including: 
30 die watermark presence being at least substantially invisible (ie the watermarked visual 
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or audio material is visually or auditorially substantially indistinguishable from the original); 

the watermark can be inserted and/or extracted by a simple computational procedure 
which can be done in real time tor audio and/or video media data; and 

the original media data not being required in order to extract the watermark from the 
S watermarked media data. 

Additionally, as also discussed above, it is desirable for watermarks in digital media data to 
also be both secure in that a malicious user cannot easily remove or disguise the watermark 
so as to prevent extraction, and robust to enable the inserted watermark to survive 
10 manipulation of the watermarked media data. Digital images and image sequences, for 
example, are seldom stored or transported over a communications link in their raw format. 
Frequendy some form of compression may be sy^plied to the media data, and it is dierefore 
important that the signal processing associated with die compression algorithm does not 
remove or wash out die associated watermark inserted in the media data. 

15 

Although the following description of embodiments of the present invention refer primarily 
to still or sequential image data, it is to be understood that die invention is equally applicable 
to other forms of digital media data, such as digitised audio data. 

20 In an embodiment of the invention, image pixel data is subdivided into 64 x 64 pixel spatial 
domain blocks in order to provide a manageable data segment in which to insert watermark 
data. For example, a digital image comprising 1,024 x 768 pixels may be nominally divided 
into blocks of 64 x 64 pixels so that the entire image is contained in an array of 16 x 12 image 
data blocks (a total of 192 data blocks). Different watermark data may tiien be inserted into 

25 each data block, so that the watermark data is spread over the entire image. For example, 
the watermark might comprise a 16 x 12 pixel logo or the like, so tiiat a value representing 
each pixel of the logo is inserted in a respective data block of the digital image. 
Alternatively, die watermark may comprise a text message formed in ASCII code and/or 
binary numbers. A message comprising of 192 bits could be inserted in the digital imaigt if 

30 a watermark density of 1/ 4096 (one bit per 64x64 block) was employed. 
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The invisibility and robustness of the watermark are aided by dividing the image into blocks 
and distributing the watermark data throughout the data blocks, and are further facilitated by 
the insertion procedure utilised to insert the watermark data into each, data block. The 
5 following st^s are used to insert a watermark data bit or binary pixel graphic into a 64 x 64 
spatial dom^n luminance data block. 

(i) Permute the 64x64 data block using a predetermined random permutation. There exist 
4096 factorial different ways in which this permutation can be performed. To 
10 minimise the distortion introduced by the watermark modification, a permutation 

should be selected that performs a spectral whitening operation on a signal that has a 
predominant low pass power spectral density. The permutation is generated from a 
keyed pseudo-random operation. 

15 (ii) Transform the permuted data using a Walsh Hadamard Transform. This transform 
can be implemented as a 4096-point one dimensional fast transform operation. 

(iii) Watermark data is inserted into the data block by modification of selected transform 
coefficient(s). The coefficient selection process is based on a keyed-pseudo random 
20 operation, and does not include the dc coefficient in set of coefficients available for 

modification. To maximise security of the watermarking process, different 
coefficients are selected via the pseudo-random operation for each data block. 

A watermark data bit can be represented by die sign of a selected transform 
25 coefficient. A transform coefficient value greater than or equal to zero could 

r^resent logic zero and the negative values logic one. Transform coefficient(s) need 
only be modified if necessary, to wisure that the sign (+/-) corresponds die digital bit 
to be embedded (1/0). 



30 (iv) 



An inverse transform is then applied to reconstruct an approximation of die original 
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64x64 spatial domain data block. In the transform domain, the watermark data is 
completely contained by one transform coefficient when using a watermark data 
density 1/4096. In the spatial domain, however, the watermark data is distributed 
over each of the pixels making up the 64x64 data block. 

5 

The watermark read operation is accomplished by repeating steps (i) and (ii) above. The 
original image or image sequence is not required for the reading operation. The watermark 
data can be extracted with the knowledge of the permutation applied to the data block, the 
transform operation, and which of the transform coefficient(s) modified to contain the 
10 watermark data. The permutation employed is preferably kept secret by the owner of the 
image or image sequence. The permutation could be represented by a secret seed number to 
a well defined pseudo random number generator. 

Block transforms such as tfie classic Walsh Hadamard Transform (WHT), Discrete Cosine 
15 Transform (DCT), Discrete Sine Transfomi (DST) and the Haar Transform (HT) can be 
employed in the watermarking process in embodiments of the invention. For transforms that 
isolate the average block value or dc value into one coefficient, that coefficient should not be 
used to contain watermark data. The WHT is the preferred choice for the transform operation 
due to its low implementation con5)lexily. Fast transform inq)lementations of the WHT exist 
20 that require only summing and one scaling operation, and the transform basis vector contains 
only +1 and -1 elements. The analysis and synthesis transforms are identical. 

Figure 1 illustrates a flow chart of operation involved in insertion of watermark data into 
digital media data, according to an embodiment of the invention. Beginning at step 12, the 

25 digital media data is first segmented into manageable data blocks such as blocks of 64x64 
pixels or equivalent data elements. Step 13 calculates the average pixel value for die block 
which is then subtracted from each pixel. Stq) 13 is unnecessary when using a transform diat 
contains the block average in a single transform coefficient. This is the case with the WHT 
and the DCT, for example. The resulting dc transform coefficient should not, however, be 

30 used to contain watermark data. The media data block or segment is then subjected to a 
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permute operation (step 14) in which the data elements of the block or segment are rearranged 
in a pseudo random, but repeatable and reversible manner. Next, at step 16, the permuted 
spatial domain media data segment is subjected to the transform operation. In this 
embodiment one of the transform coefficients is selected and modified to include watermark 
S data. When watermarking images or image sequences a watermark data bit could be 
rqaresented by the sign of the selected transform coefficient. A transform coefficient value 
greater than or equal to zero could represent logic zero and the negative values logic one. 

The watermark data density per block in this case is 1/4096. In some applications, densities 
10 greater than 1/4096 may be required. 

Following insertion of the watermark data into the transform domain of the media data, the 
spatial domain media data is then reconstructed through steps 20, 22 and 23 by performing 
an inverse transformation followed by an inverse permute operation and then the previously 

15 subtracted block average value added to each pixel making up the block. Again, step 23 is 
not necessary when usmg a transform that contains the block average in a single transform 
coefficient The resulting digital media data segment contains watermark data which is robust 
to manipulation thereof, secure from unauthorised removal, and yet the reconstructed, 
watermarked media data is substantially indistinguishable from the original spatial domain 

20 media data when compared in subjective quality testing. 

In order to extract the watermark data form digital media data in which watermark data has 
been previously inserted, the procedure oudined in the flow chart of Figure 2 may be 
employed. Essmtially this involves steps mirroring the first half of the procedure illustrated 

25 in Figure 1. The digital media data is first segmented as discussed previously (step 32), the 
average pixel value for that block is determined and subtracted from each pixel (step 33) if 
necessary. The resulting data block is then subjected to a permute operation as shown at step 
34. The permute operation must be the same as that performed during insertion of the 
watermark data, and thus if different permute operations are variously employed, some 

30 record must be maintained of which of the particular 4096 factorial permutations s^plies to 
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the particular media data segment in question. This could be in the form of a secret seed to 
a well defined pseudo random number generator. The permuted media data segment is then 
transformed with the same transform used by the insertion operation (step 36). Then it is a 
simple matter to extract the particular coefficient for the transform domain media data and 
5 then recover from diis the watermark information. 

Figure 3 illustrates a block diagram of the watermark insertion process described in 
connection with the flow chart of Figure 1. As discussed above, in this embodiment only a 
single watermark data component, eg a data bit or binary graphic pixel, is inserted into each 
10 selected digital media data segment or block, and the information required to reconstruct an 
entire watermark requires the examination of a number of digital media data segments. 

Figure 4 is a flow chart illustrating the insertion process of watermark data into digital media 
data, which has been segmented into data blocks, over a series of data blocks. Where the 

1 5 digital media data comprises a sequence of images, such as in the case of digital video or the 
like, a complete watermark (eg the total of the identification data) may in fact be distributed 
over more than one image or image frame. At step 42 the first data block in the image or 
sequence of images is selected and, if necessary, the average of that block is tiien calculated 
and subtracted from each pixel element in step 43. The resulting data block forming die 

20 image segment is subjected to a permute operation, as described hereinabove, at st^ 44. The 
permuted image data is then transformed using a block transform. At step 47 a particular 
transform coefficient is selected for possible modification. The selection process is performed 
in a psaido random deterministic manner. Transforms tiiat contain die block average (dc) in 
one transform coefficient, or set of coefficients, must elinunate this coefficient from the 

25 selection process. Step 48 performs the modification operation to incorporate the watermark 
data into the selected transform coefficieni(s). The inverse of the transformation and permute 
operations are then applied at steps 50 and 52 and step 53 adds to each pixel value the average 
as determined in step 43, if necessary. A test is then ^plied at step 54 to determine whether 
die media data has finished, and if so die watermarking procedure ends. Otherwise, the next 

30 block of the digital media data is selected at step 56. The watermark data is dien 
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incremented, meaning the next component of the watennark data, such as the next data bit 
or binary pixel element, is selected at step 58. Of course, it will be recognised that it is 
unnecessary for every data block of a particular digital media data source to be encoded with 
watermark data, and only a certain selection of data blocks may in fact be encoded with 

5 watermark data in practice. To provide copyright protection for the complete image 
sequence, the watennark can be repeatedly inserted, with the watermark beginning at different 
frame locations within the sequence and ensuring that watermarks do not overlap. Of course, 
acquisition of the signal is important. This can be accomplished, by incorporating in the 
watermark data, synchronisation informauon that, once acquired informs the watennark 

10 reader the location of the beginning of the watermark message data or binary gr^hic. 

To increase robusmess and ensure readability even in the case where the original video signal 
is significantly changed, such as through reduced spatial resolution or the case where 
watermarked interfaced material is later converted to non-interlaced format, the watermark 

15 can be distributed across both fields in such a way that the watermark can be independently 
read from either or both fields and/or restricted to the low ^atial frequencies. The latter may 
be acconqjlished by the application of a 2x2 WHT on each row of the image to produce low 
and high spatial frequency components. The watermark is then inserted in only the half 
horizontal resolution frame corresponding to die low spatial frequencies. The full resolution 

20 watermarked frame is produced by performing an inverse 2x2 WHT on the rows making up 
the low spatial frequency watermarked half horizontal resolution frame and the original high 
spatial frequency half horizontal resolution frame. 

In order to fiirdier improve security of die watermarkmg procedure, it is possible to alter the 
25 permute operation periodically (step 60 in Figure 4). As mentioned above, it is nevertheless 
necessary that the particular permute operation performed on each data block be repeatable 
at a future time to enable extraction of die watermark. 

Figure 5 illusu-aites a block diagram of watermarking apparatus for encoding real time video 
30 with watermark data according to an embodiment of the present invention. Real time video 
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feed is provided to the apparatus at a buffer 80 or the like, which provides an input to real 
time processing circuitry 82. The circuitry 82 may comprise digital processing circuitry in 
the fomi of high speed programmable computer circuitry, for exanple, which carries out the 
algorithmic steps described in connection with Figure 4, for exanq)le. The watermark data 
5 is provided from a buffer 84 which may be in the form, for example, of a ring buffer which 
cyclically feeds watermark data being a component of watermark text or graphic material to 
the processing circuitry 82. The reconstructed video data containing the watermark data is 
then passed to an ouq}ut buffer 86 which provides the video data for transmission, recording 
or whatever function the video data is required for. 

10 

Embodiments of the invention, operating in real time, can be utilised to add watermark data 
to media such as video and/or audio during live broadcast or other transmission, whilst 
recording to storage such as tape or disc, during broadcast or other transmission from storage, 
and during transferral from one storage device to another, for example. Furthermore, 

15 embodiments of the invention operating in real time can be used to monitor media such as 
television transmissions to detect the presence of watermark data incorporated in the media 
data. A block diagram of such a system is illustrated in Figure 6. Video data is provided to 
a buffer 90 from a source such as a broadcast receiver or the like. Real time processing 
circuitry 93 is coupled to receive the media data from the buffer 90 and perform the 

20 algorithmic steps described in connection with Figure 2, for example. This results in the 
extraction of any watermarking data contained in the media data which was inserted according 
to a process known to the monitoring apparatus (i.e. watermark data which has been added 
with a known permutation and transform in transform coefficients selected according to a 
known scheme). A comparison processor 94 can then be \ised to compare any watermaric data 

25 which is retrieved with stored watermark data to determine if the retrieved watermark data 
corresponds to a known watermark indicating the source of the media data. 

It will be appreciated from the foregoing description that the original media data is not 
required by the watermark extraction process in order to extract the watermark data, and 
30 therefore it is not required that the original image be certified by a tmsted third party or held 



wo 98/37513 



PCT/AU98/00106 



. 16- 

in escrow in order to prove the presence of a watermark in the media data. Random 
accessibility of a watermark within an image sequence is easily achieved, as all that is 
required to extract the watermark is the image or sequence of images that contains sufficient 
watermark data to reconstruct the entire watermark or a substantial portion thereof, and the 
S secret keys used to seed the random permutation and the random coefficient selection process. 

The watermarking process according to an embodiment of the invention has been tested on 
still images and image sequences, and has been demonstrated to be near invisible to the naked 
eye in a comparison between tiie reconstructed, watermarked media data and die original 

10 media data. It has also be found to be secure and robust to conyiression such as 4 Mbps 
MPEG coding of image sequences and 20% quality setting for JPEG compressed still images. 
The described watermarking procedure is also robust to digital-to-analogue and analogue-to- 
digital conversions. Accordingly, embodiments of the invention can be utilised to insert and 
extract watermark data in analogue media as well as digital media. For example, watermark 

15 data can be inserted and extracted from broadcast or home quality analogue or digital video. 
Tests have been performed demonstrating a successful read operation for watermarked digital 
video originally of broadcast studio quality which has been temporarily recorded on an 
analogue consumer VHS tape. In the case where the media is generated, stored and/or 
transmitted in an analogue form, an analogue-to-digitai conversion using known techniques 

20 is used to obtain digital media data before inserting or extracting the watermark data (see 92 
in Figure 6). The media data may be returned to analogue form, if desired, using known 
digital-to-analogue techniques. 

It will also be appreciated that die simple nature of the computational processes involved in 
25 the watenmaiidng process of die present invention allow it to be a^)plied quite readily to real 
time video data, for example. This is because the only two computationally complex steps 
in the watermaricing procedure, namely the permute and transformation are still relatively 
simple. This makes for a watermarking process that is very low in complexity, is easily 
automated, and requires no human intervention in its application. 

30 
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The foregoing detailed description of the present invention has been presented by way of 
example only, and is not intended to be considered limiting to the invention as defined in the 
claims appended hereto. 



5 
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Qaimsi 

1. A method for inserting identification or authentication data into digital media data, 
including the steps of: 

5 segmenting the digital media data into data blocks; 

applying a pseudo-random reversible function to a block of the digital media data to 
obtain a modified data block; 

applying an orthogonal transform on the modified data block to obtain transform 
domain data; 

10 modifying at least one selected transform domain data coefficient in accordance with 

identification or audienucation data; 

inverse transforming the transform domain data having the at least one modified 
coefficient; and 

applying an inverse pseudo-random function to obtain watermarked digital media data. 

15 

2. A metiiod as claimed in claim 1, wherein the pseudo-random function j^plied to the 
data block is a keyed function controlled by a cryptographic key. 

3. A method as claimed in claim I or 2, wherein the pseudo-random function applied to 
20 the data block has a property of flattening the power spectral density of the data block. 

4. A method as claimed in claim 1, wherein application of the pseudo-random function 
and application of the orthogonal transform are carried out in the same operation. 

25 5. A method as claimed in claim 1, wherein the at least one transform domain data 
coefficient selected for modification is selected according to a keyed pseudo-random 
operation. 

6. A method as claimed in claim 1, wherein a pluraliQ^ of data blocks of the digital media 
30 data are modified according to the identification or authentication data. 
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7. A method as claimed in any one of claims 1 to 6, wherein the digital media data is 
video data. 

8. A method as claimed in any one of claims 1 to 6, wherein die digital media data is 
5 audio data. 

9. A method as claimed in clam 7 or 8, wherein the identification or authentication data 
is insened into the digital media data in real time. 

10 10. A method as claimed in claim 1, wherein at least one coefficient in the transform 
domain data which represents the average (dc) of the data block is restricted from selection 
for modification with the identification or audientication data. 

11. A method as claimed in claim 1 or 10, wherein the orthogonal transform is a Walsh 
15 Hadamard transform. 

12. A metiiod as claimed in claim 1 or 10, wherein die ortiiogonal transform is selected 
from a discrete cosine transform, a discrete sine transform and a fast Fourier transform. 

20 13. A method as claimed in claim 1, wherein the pseudo-random reversible function is a 
permutation of the data block based on a keyed pseudo-random number generator, 

14. A method as claimed in claim 1 , including determining an average of data values in 
the data block, subtracting tiie average value from the data values in the data block before 

25 applying the pseudo-random tunction, and adding the average value back to the data values 
in the data block after applying the inverse pseudo-random function. 

15. A method for extracting identification or authentication data from watermarked digital 
media data, including die steps of: 

30 segmenting the digital media data into data blocks; 



wo 98/37513 



PCT/AU98/00106 



-20- 

applying a pseudo-random reversible ftinction to a block of the digital media data to 
obtain a modified data block; 

applying an orthogonal transform to the modified data block to obtain transform 
domain data; and 

5 extracting identification or authentication data from at least one coefficient of the 

transform domain data. 

16. A method as claimed in claim 15, wherein the pseudo-random function applied to the 
data block is a keyed tunction controlled by a cryptographic key. 

10 

17. A method as claimed in claim 15 or 16» wherein the pseudo-random function s^piied 
to the data block has a property of flattening die power spectral density of the data block. 

18. A method as claimed in claim 15, wherein application of the pseudo-random function 
15 and application of the orthogonal transform are carried out in the same operation. 

19. A method as claimed in claim 15, wherein the extracting step includes selecting at 
least one transform domain data coefficient from which to extract identification or 
authentication data according to a keyed pseudo-random operation. 

20 

20. A method as claimed in any one of claims 15 to 19, wherein the digital media data 
comprises video data, 

21. A method as claimed in any one of claims 15 to 19, wherein the digital media data 
25 conq)rises audio data. 

22. A method as claimed in claim 20 or 21 , wherein the identification or authentication 
data is extracted from the digital media data in real time. 

30 23. A method as claimed in claim 15, wherein the orthogonal transform is a Walsh 
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Hadamard transform. 

24. A method as claimed in claim 15, wherein the orthogonal transform is selected from 
a discrete cosine transform, a discrete sine transform and a fast Fourier transform. 

5 

25. A mediod as claimed in claim 15, wherein the pseudo-raiKiom reversible function is 
a permutation of the data block based on a keyed pseudo-random number generator. 

26. A method as claimed in claim 15, including determining an average of data values in 
1 0 the data block, and subtracdng the average value from the data values in the data block before 

applying the pseudo-random function. 

27. An apparatus for inserting or extracting watermark data in digital media data, 

comprising: 

15 segmenting means for segmenting the digital media data into data blocks; 

processing means for applying a pseudo-random reversible function to a block of the 
digital media data to obtain a modified data block and performiiig a transfoim on the modified 
data block to obtain transform domain data; and 

means for inserting or extracting watermark data in at least one coefficient of the 
20 transform domain data. 

28. An apparatus as claimed in claim 27, wterein the processing means is also adapted to 
apply an inverse transformation and inverse pswdo-random function of the transform domain 
data containing die watermark data so as to generate watermarked digital media data. 

25 

29. An s^params as claimed in claim 27 or 28, wherein die apparatus inserts or extracts 
watermark data in digital media data in real time 



30 



30. 
data. 



An apparatus as claimed in claim 29, wherein the digital media data comprises video 
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31 . An apparatus as claimed in claim 29, wherein the digital media dam comprises audio 
data. 

32. An apparatus as claimed in claim 27, including means for selecting at least one 
5 transform domain data coefficient for the insertion or extraction of ' identification or 

authentication data according to a keyed pseudo-random operation. 

33. A media data monitoring system comprising: 

a media data buffer for temporarily storing media data received from a data source; 
10 a real time processor coupled to receive media data from the media data buffer and 

adapted to extr^t identification or authentication data according to die method defmed in 
claim 15; and 

a comparison processor coupled to the real time processor for comparing extracted 
identification or autiientication data wiUi known identification or authentication data. 

15 

34. A media monitoring system as claimed in claim 33, including an analogue-to-digital 
converter for converting media data into a digital form before processing by the real time 
processor. 

20 35. A media monitoring system as claimed in claim 33 or 34, wherein the media data 
comprises video data. 

36. A media monitoring system as claimed in claim 35, wherein die data source of the 
media data is a receiver of video transmissions. 

25 

37. A media data monitoring method comprising: 
receiving media data from a data source; 

extracting identification or authentication data according to the method defined in 
claim 15; and 

30 comparing extracted identification or authentication data with known identification or 
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authentication data. 

38. A media monitoring method as claimed in claim 37, including convening the media 
data into a digital fomi before processing by the real time processor. 

5 

39. A media monitoring method as claimed in claim 37 or 38, wherein the media data 
compriseis video data. 

40. A media monitoring method as claimed in claim 39, wherein the media data is 
10 received from a video transmission. 
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